iiDiiiniiiniiiiHiiui 

US006404818B1 

(12) United States Patent (h» Patent No.: us 6,404,818 bi 

Obikane (45) Date of Patent: Jun. 11, 2002 



(54) VIDEO TRANSMISSION DEVICE AND ITS 
METHOD 

(75) Inventor: Nobuhisa Obikane, Kanagawa (JP) 

(73) Assignee: Sony Corporation, Tokyo (JP) 

( * ) Notice: Subject to any disclaimer, the term of this 
patent is extended or adjusted under 35 
U.S.C 154(b) by 0 days. 

(21) Appl. No.: 09/234,169 

(22) Filed: Jan. 19, 1999 

(30) Foreign Application Priority Data 

Jan. 20, 1998 (JP) 10-009094 

(51) Int. CI. 7 H04N7/12 

(52) U.S. CI 375/240.28; 348/500 

(58) Field of Search 386/125, 126, 

386/95, 109; 375/240.28; 348/500, 505, 
521, 545, 547, 467, 466, 390, 461, 423.1; 
370/389, 401, 468, 498; H04N 7/12 



(56) References Cited 
U.S. PATENT DOCUMENTS 

5,517,250 A * 5/1996 Hoogenboom et al 348/467 

5,621,840 A * 4/1997 Kawamura et al 386/68 

5,732,185 A * 3/1998 Hirayama et al 386/70 

5,856,973 A * 1/1999 Thomson 370/389 

5,884,004 A * 3/1999 Sato et al 386/98 

* cited by examiner 

Primary Examiner — Nhon Diep 

(74) Attorney, Agent, or Firm — Frommer Lawrence & 
Haug LLP; William S. Frommer 

(57) ABSTRACT 

In a video transmission device, a plurality of video data 
associated with one program can be transmitted without a 
complicated processing at the time of reception. In the case 
of transmitting a plurality of associated video data, first 
program information to control the plurality of associated 
video data as one program is formed and transmitted, so that 
the plurality of video data associated with one program can 
be transmitted without a complicated processing. 

8 Claims, 10 Drawing Sheets 



SVL- 



o 
po 

o 

tu 

no 
PO 
o 



SVH- 



"-o 

PO 

o 

PO 



SVH - 



CO 



Sa 



S33- 



22 



23 



VIDEO 




ENCODER 





24 - 
\ r i 



ENCODING 
CONTROL 
UNIT 



26 



21B 



S22 



PID=XV1 



PACKET I ZER 



21C- 



TS 

PACKET 1 ZER 



VIDEO 




ENCODER 





22B 



S25 



PID-XV2 



PACKET I ZER 



T2C 



TS 

PACKETIZER 



AUDIO 




ENCODER 








^24A 


S30 


AUDIO 




ENCODER 





24B 



PACKETIZER 



S31 
1 



PID-XV3 



23C~ 



TS 

PACKETIZER 



23B 



PACKETIZER 



S28 
1 



PID-XA 



24C 



TS 

PACKETIZER 



S23 
/ 



S26 



S29 



S32 



PCR 



PID=XC S34 



.12 



TS 

PACKETIZER 



27MHZ 
CLOCK 




STC 


\ J 



'11 



T 

27 



PIfrXX 



20 



- 25 
S36 



PID=0 



.S40" 



PAT 
FORMATION 



28b' 

TRANSPORT , 

STREAM JS37J 
OF OTHER S §§§= 

PROGRAM ^ y 



29 
_1_ 



x 
rn 



S41> 



-S35 



PMT 
FORMATION 



[TRANSMISSION 
UNIT 



-28A 



T 

S42 



■wlO 



CLK1 



03/03/2004, EAST Version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 1 of 10 US 6,404,818 Bl 



TRANSMISSION 




RECEIVING 


— > 


RECEIVER 


DEVICE 


—2 


DEVICE 





FIG. 1 (RELATED ART) 



SI r 



L 

VIDEO 
ENCODER 



S2 



AUDIO 
ENCODER 



S3 



7A 

I 

PACKET I ZER 



6 

L- S5 

-L 



7B 



PACKET I ZER 



S4 
1* 



1Q 



S6 



Cxi 
X 

w 



-8 
S7 



jL 




TRANSMISSION 
UNIT 



S8 



FIG. 2(RELATED ART) 



03/03/2004, EAST version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 2 of 10 US 6,404,818 Bl 



FIG. 3A 
(RELATED ART) 



si 



S3 



GOP 



FIG. 3B 
(RELATED ART) 



ES PACKET HEADER 



GOP 



GOP 



GOP 




COMPRESSION CODED DATA 



FIG. 4A 
(RELATED ART) 



S3 



S4 



FIG. 4B 
(RELATED ART) 




PES HEADER 

V y ' 

PES PACKET 



03/03/2004, EAST Version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 3 of 10 US 6,404,818 Bl 



FIG. 5A(R EL ATED ART) 

PES PACKET I PES HEADER | DATA 
TS PACKET U J U U U L 

T I 

TS HEADER PES DATA 

FIG. 5B(RELATED ART) 



FIG. 6A(RELATED ART) 



1 TS PACKET- 
SYNCHRONIZING SIGNAL 



DATA ENABLE ~\_ 



iTS HEADER 



1 1 




TS PACKET XXXXXXXX.'! 

i i 



1 1 

-n- 



PES DATA 



4byte 184byte 

FIG. 6B(RELATED ART) 



03/03/2004, EAST version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 4 of 10 US 6,404,818 Bl 



PROGRAM NUMBERHO: - 


P I D 


= N 


PROGRAM NUMBERHU - 

• 
• 


-^PID 

• 
• 


= A 


• 
• 

PROGRAM NUMBERUX] - 


• 

-^PID 


= XX 



FIG. 7(RELATED ART) 



PROGRAM NUMBER X 

PID OF VIDEO DATA=X V 

PID OF AUDIO DATA=XA 

PID OF TEXT DATA=XD 

PID OF CLOCK INFORMATION=XC 



FIG. 8(RELATED ART) 



03/03/2004, EAST Version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 5 of 10 



US 6,404,818 Bl 



RECEIVED 
TRANSPORT 
STREAM 



PID=0 
RECEPTION 



PID=XX 
RECEPTION 



PID=XV 
RECEPTION 



PROGRAM NUMBER 0->PID=N 
PROGRAM NUMBER 1— PID=A 

PROGRAM NUMBER X-^PID=XX 



PID=XA 




-5» 


RECEPTION 







PID=XD 






RECEPTION 




-> 



PROGRAM NUMBER X 
VIDEO PID=XV 
AUDIO PID=XA 
DATA PUHXD 
CLOCK PID=XC 



VIDEO DATA DECODING 



AUDIO DATA DECODING 



DATA DECODING PROCESSING 



PID=XC 




RECEPTION 


— > 



RECOVERY CLOCK GENERATION 



FIG. 9(RELATED ART) 



03/03/2004, EAST Version: 1.4.1 



U.S. Patent 



Jun. 11, 2002 Sheet 6 of 10 



US 6,404,818 Bl 



I— t 



CD 

5z; 



24MHZ 
CLOCK 


r 


STC 


\' 


PACKET I ZER 



12 



11 



OTHER TS PACKET 




PID=XC 
RECEPTION 



13 



PCR 
14- 



15 



STC 



PLL 



27MHZ 
CLOCK 



CLK2 



16 l... 3 



FIG. 10(RELATED ART) 



13 



PID=XV 
RECEPTION 



18 



PTS 
RECEPTION 



17 



* DECODER 



15- 



STC 



i_n 



DATA 



19 



TIME 
COMPARATOR 



FIG. 11 (RELATED ART) 



03/03/2004, EAST version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 7 of 10 US 6,404,818 Bl 



ot 



en 



MULTIPLEXER 



CO 
CO 




C\J 

CD 

Ll. 



DATA FOR ONE PROGRAM (PROGRAM NUMBER X) 



03/03/2004, EAST version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 8 of 10 



US 6,404,818 Bl 




cu 
o 

CD 



a 


CQ 


i— t 


PQ 


00 

rH 


CU 


CV 

rH 


PQ 


CO 

rH 


CQ 


LO 
rH 


Cu 


rH 


CQ 


00 

rH 


CQ 


CM 

rH 


Cu 


rH 
rH 


CO 


o 

rH 


CQ 


o> 


CU 


00 


CQ 




PQ 


CD 




LO 


PQ 


^ 


CQ 


CO 


Oh 


<N) 


CQ 


rH 


PQ 




o 





CQ 


rH 


CQ 


CO 

rH 


CU 


t> 
rH 


CQ 


CO 

rH 


PQ 


to 

rH 


PU 


rH 


CQ 


00 

rH 


CQ 




Ou 


rH 
rH 


PQ 


O 
rH 


CO 


o> 


Cu 


CO 


PQ 




PQ 


CD 


» i 


LO 


PQ 


xt* 


PQ 


00 


OU 


CM 


PQ 


rH 


PQ 



LO 

rH 

II 



CO 
li 



oo 

>■ 
GO 




>■ 

C/0 



< 

CO 



GO 
CO 



O 

CO 



CD 

Li_ 



CD 



CD 



03/03/2004, EAST Version: 



1.4.1 



U.S. Patent Jun. 11,2002 Sheet 9 of 10 US 6,404,818 



PROGRAM NUMBER X 

PID OF LOW ANGLE VIDEO DATA = XVI 
PID OF MIDDLE ANGLE VIDEO DATA = XV2 
PID OF HIGH ANGLE VIDEO DATA = XV3 
PID OF AUDIO DATA = XA 
PID OF CLOCK INFORMATION = XC 



FIG. 14 



03/03/2004, EAST version: 1.4.1 



U.S. Patent Jun. ll, 2002 Sheet 10 of 10 US 6,404,818 Bl 




^2. 




in 

* CO 



STC 






PLL 








03/03/2004, EAST version: 1.4.1 



US 6,404,818 Bl 

1 2 

VIDEO TRANSMISSION DEVICE AND ITS samples called as an audio frame in the same manner as in 

METHOD me case °f video data shown in FIG. 2, and by adding a 

header comprising parameter information at the time of 

BACKGROUND OF THE INVENTION encoding to the head of the coded audio data, forms an 

5 ES-packetized coded audio data S5. 

1. Field of the Invention Moreover, the packetizer 7B, by connecting a fixed num- 
The present invention relates to a video transmission ber of ES packets of the coded audio data S5 encoded per 

device and its method and more particularly, is suitably audio frame and adding the PES header to the head of the 

applied to a digital broadcasting system. packet, PES-packetizes the coded audio data S5 and outputs 

2. Description of the Related Art 10 jhePES-packetized coded audio data S6 to the multiplexer 

.... e . , , 8. The data of the PES header are approximately the same as 

A digital broadcasting system for multiplexing coded ^ tbe case 0 £ v jdeo 

video and audio data to transmitting them through a satellite ^ mulli lexer 8 tra n S port-stre a m (TS)-packetizes the 

circuit has been proposed and become practical use. In this PES .packetized coded video data S4 and coded audio data 

digital system, video data and also audio data are encoded S6 b dividi lhem fixed dala unit respectivelV) and by 

and by packeUzing and multiplexing the coded video and is swi[chin and outputting the packets of the ^ aod 

audio data per fixed data unit, transport data (hereinafter, audio at established mtervals> mu i tip ie X es them to form a 

referred to as a transport stream) is formed and transmitted transport stream S7 

through such as a satellite wireless circuit. KA . a „ ' , . CI ™ CA -„ A CTi 

e More specifically, as shown in FIGS, 5 A, 5B, 6 A and 6B, 

Here, the digital broadcasting system will be described the multiplexer 8„by dividing each PES packet of the coded 

with reference to FIG. 1. As shown in FIG. 1, a digital video data s4 and audio data S6 per 184 bytes and 

broadcasting system 1 is roughly comprised of a transmis- a dding a TS header of 4 bytes to each of their heads, forms 

sion device 2 to be set in a broadcasting station, a receiving a TS packet In this casCy the j$ neader ^ formed of 

device 3 and a television receiver 4 to be installed in each synchronization information and packet identification infor- 

house. Transport stream data to be transmitted from the mat i 0 n PID showing the type of data included in the TS 

transmission device 2 is received by the receiving device 3, packet HoweV er, the data part of 184 bytes is not only 

and by sending out this to the television receiver 4, broad- comprised of PES data but in practice, also additional 

casting video and audio can be watched. information is added and these additional information and 

As shown in FIG. 2, in the transmission device 2, firstly, PES data are adjusted to become 184 bytes altogether. Such 

video data SI to be transmitted is entered into a video 3Q data adjusting processing is generally called as a stuffing 

encoder 5, and audio data S2 accompanying with the video processing. In this connection, even when PES data are not 

data SI is entered into an audio encoder 6. The video encoder enough, the stuffing processing is conducted so that data part 

5 successively encodes the entered video data SI based on becomes 184 bytes by adding dummy byte. Then, the 

the MPEG2 video encoding system described in the Inter- multiplexer 8 forms a transport stream S7 by multiplexing 

national Standards Regulation ITU-T Recommendation 35 thus formed video TS packets onto thus formed audio TS 

H.262 ISO/IEC13818-2, and outputs a stream of the result- packets. 

ing coded video data S3 (generally, this is called as an i n this connection, since proper packet identification 

elementary stream (ES)) to a packetizer 7A. In the case of information PID is added to the multiplexed video and audio 

encoding the video data SI, the video encoder 5 applies an TS packet per data, the receiving side can separate the TS 

encoding processing with video data for several pictures, 4Q packets based on the packet identification information PID 

which are called as group of picture (GOP), as a coding unit, l0 decode video and audio from one multiplexed transport 

and by adding a header consisting of each parameter infor- stream. 

mation at the time of encoding to the head of the coded video the ^sport stream S7 comprised of video and 

data, forms coded video data S3. Thus, a decoding process- audio ^ packets ^ transmitted to the succeeding transmis- 

ing can be conducted based on the parameter information of 45 sion unit 9 ^ transmiss i 0D unit 9, after forming a trans- 

the header at the time of decoding. mission signal by app i ying the modulation processing to the 

As shown in FIGS. 3A and 3B, the packetizer 7A, by transport stream S7 for transmitting, forms a transmission 

connecting a fixed number of ES packets of coded video data signal S8 by applying a frequency conversion processing to 

S3 encoded per GOP and adding a packetized elementary the transmission signal of a stated band and transmits this 

stream (PES) header to the head of the packet, PES-packets 50 toward the satellite (not shown) via an antenna 10. 

the coded video data S3 and outputs the PES-packetized a transmission signal S8 relayed via the satellite is 

coded video data S4 to a multiplexer 8. Note that, data of the received by the receiving device 3 shown in FIG. 1 and the 

PES header consist of time information when reproducing receiving device 3 conducting a reverse processing of the 

and outputting the video data included in the PES packet at transmission device 2, so that the transmission signal S8 is 

the receiving side (generally, called as presentation time 55 decoded to the original video and audio data. More 

stamp (PTS)), identification information showing the type of specifically, the receiving device 3 applies the frequency 

elementary stream included in the PES packet (generally, conversion processing and demodulation processing to the 

called as a stream ID), and the like. In this connection, such received signal to restore the transport stream S8 transmitted 

time information PTS is provided in the PES header, so that fr om tne transmission device 2. And then, the receiving 

audio and video can be synchronized and reproduced at the 60 device 3 separates video TS packets and audio TS packets 

receiving side. based on the packet identification information PID and takes 

On the other hand, the audio encoder 6 encodes the input out coded video data and coded audio data from each TS 

audio data S2 based on an audio encoding system described packet separated, and applying the decoding processing to 

in the International Standards Regulation ITU-T Recom- these data, restores video data and audio data transmitted, 

mendation H.262 ISO/IEC13181-3 and outputs the resultant 65 Thus, these restored video data and audio data are transmit- 

coded audio data S5 to a packetizer 7B. Note that, the audio ted to the television receiver 4, so that video and audio of the 

encoder 6 encodes the audio data S2 per fixed number of program sent from the transmitter 2 can be watched. 
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In the foregoing explanation, the transmission device 2 program information PAT and PMT will be described below 

transmits video and audio data forming one program. with reference to FIG. 9. First, in the receiving device 3, 

However, in practice, the transmission device 2 transmits after turning the power source ON, the TS packet having the 

video and audio data of the plural number of programs in the packet identification information PID "0" is extracted from 

form of one transport stream S8 by TS-packetizing and 5 the transport stream received to obtain the program infor- 

multiplexing video and audio data of plural programs. mation PAT. At this point, if a request to display the program 

Furthermore, in the case where text data are attached to the having the program number "X" is entered by the user, the 

video data, the text data can also be transmitted in the form receiving device 3, referring to this program information 

of one transport stream S8 by being TS-packetized and PAT, investigates the packet identification information PID 

multiplexed. In this connection, if the plural programs are 1Q ofTS packet in which the program information PMT having 

multiplexed, the receiving side extracts TS packets of a the program number "X" is stored. In this case, since the TS 

desired program based on the packet identification informa- packet in which the program information PMT having the 

lion PID added to the TS packets and thus, the video and program number "X" is stored has the packet identification 

audio data of the desired program can be restored. information PID "XX", the receiving device 3 extracts the 

At * ;„ ™« „f mi ,n^w;™ TS packet having the packet identification information PID 

At this point, the processing in the case or multiplexing ,c c .i_ . _ * * j . • *u 

t , . , r r . , -i j • j * *i *u 15 XX from the transport stream received to obtain the 

the plural programs will be described in detail in the . - nxjrr *u _~ ~u 

, . * ■ • • j .... program information PMT concerning the program number 

following paragraphs. Since the receiving side extracts the £^,f 

TS packets of the desired program based on the packet ^ ^ ^ ^ tQ ^ m 

identification information PID as . described above, it is mformat ; on PMT obtained) can grasp the packet m 

necessary to know the packet identification information PID 20 which each daU associated ^ the program number « x » is 

added to the TS packet of video and audio data of each st0fed me receiving device 3 obtains and decodes 

program. Accordingly, the transmission device 2 coded video data having the program number "X" by 

TS-packetizes the program information showing the packet extracting the TS packet having the packet identification 

identification information PID added to the TS packet of information PID "XV" from the transport stream received, 

each program, and multiplexes and transmits them onto the 25 Simultaneously, the receiving device 3, by extracting the TS 

video and audio TS packets. packet having the packet identification information PID 

In this case, the program information to be transmitted is "XA" from the transport stream, obtains and decodes the 

formed hierarchically, and consists of program information coded audio data having the program number "X". 

of higher hierarchy called as program association table Moreover, in tandem with the video and audio decoding 

(PAT) and program information of lower hierarchy called as 30 processing, the receiving device 3, by extracting the TS 

program map table (PMT), As shown in FIG. 7, the higher packet having the packet identification information PID 

hierarchy program information PAT is the information to "XD" from the transport stream received, obtains and 

show the packet identification information PID of TS packet decodes text data on the program number "X". 

in which the program information PMT formed per program Simultaneously, by extracting the TS packet having the 

is stored, that is, the information like a table of contents on 35 packet identification information PID "XC", the receiving 

the program information PMT. Referring to the program device 3 obtains clock information on the program number 

information PAT, it is possible to know in which TS packet "X" and generates a recovery clock to become the reference 

the program information PMT of each program is stored. For for decoding processing and display-output based on the 

example, it is clear that the program information PMT clock information. In this connection, the recovery clock is 

having the program number "0" is stored in the TS packet 40 necessary for decoding video data and audio data, in the 

having the packet identification information PID "N". Note strict sense, the generating processing of the recovery clock 

that, "0" is added to the TS packet storing the program is to be started first. 

information PAT as the packet identification information Accordingly, the receiving device 3, firstly, obtains the 

PID. Thus, the receiving side, extracting the TS packet program information PAT which is equivalent to a table of 

whose packet identification information PID is "0" from the 45 contents, and then referring to the program information PAT, 

transport stream received and decoding it, can obtain the obtains the program information PMT showing the details of 

program information PAT. a program requested by a user and referring to the program 

On the other hand, as shown in FIG. 8, the program information PMT, obtains coded video data and coded audio 

information PMT is the information to show the packet data concerning the requested program. With this 

identification information PID of TS packet in which each 50 arrangement, even in the case where multiple programs are 

data forming the program is stored, per program and is the multiplexed in one transport stream, the requested program 

detailed program information formed per program. can be reproduced and sent out. 

Accordingly, by referring to the program information PMT, At this point, the generating processing of recovery clock 

it is possible to know in which TS packet each data forming described above will be explained more concretely. The 

the program is stored. For example, it is clear that the video 55 generating processing of the recovery clock is generally 

data having the program number "X" is stored in the TS called as PCR recovery and is for recovering the clock used 

packet having the packet identification information PID in the transmission device 2, in the receiving device 3. In the 

"XV", and the audio data having the program number "X" transmission device 2, the encoding processing for video 

is stored in the TS packet having the packet identification data and audio data are conducted in synchronism with 27 

information PID "XA". Further, the text data having the 60 MHz clock. In the receiving device 3, the same clock as the 

program number "X" is stored in the TS packet having the clock at the time of the encoding processing is reproduced 

packet identification information "XD", and the clock infor- and the decoding processing and the display-output are 

mation having the program number "X" is stored in the TS conducted at the timing specified by the transmission device 

packet having the packet identification information PID 2, in order to correctly reproduce video data and audio data 

"XC". 65 without excess or lack of data. 

At this point, an algorithm in the case of selecting a Accordingly, in the digital broadcasting system 1, clock 

desired program in the receiving device 3 based on the synchronization is to be conducted by the processing 
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described as follows in the receiving side. As shown in FIG. extracts time information PTS from the PES packet and 
10, a counter 11 called as a system time clock (STQ is transmits this to a time comparator 19. The count value of 
provided in the transmission device 2. This counter (STC) 11 the counter (STQ 15 is also supplied into the time com- 
is formed of 33-bit counter, and by conducting the counting parator 19, and the time comparator 19 compares the count 
operation with 27 MHz clock CLK1, which is generated in 5 value of the counter (STC) 15 to the value of time infer- 
tile clock generator 12 and becomes the reference clock for mation PTS supplied from the PTS receiving unit 18. If 
the encoding processing, can count 24 hours. In the trans- these count values coincide, the timing information showing 
mission device 2, making the count value of the counter the decoding start timing is transmitted to the decoder 17. 
(STC) 11 as clock information PCR, the packetizer 7C With this arrangement, in the decoder 17, by starting to 
TS-packetizes at a fixed cycle. The multiplexer 8 multi- 10 decode the PES packet based on this timing information, the 
plexes and transmits the TS packet consisting the clock decoding processing can be conducted at the timing sped- 
information PCR together with other TS packets such as fied by the time information PTS and the video data can be 
video and audio. Note that, the packet identification infer- output at the timing in synchronism with the audio data, the 
mation PID of the clock information to be regulated by the occurrence of excess or lack of data can be prevented, 
aforementioned program information PMT is added to the 1S Note tnat> m trje case 0 f video data, a horizontal synchro- 
TS packet including the clock information PCR. nizing signal and a vertical synchronizing signal are needed 
Accordingly, the receiving device 3, by referring to the when displaying video. These horizontal and vertical syn- 
program information PMT, can extract the TS packet in chronizing signals are to be formed based on the timing 
which the clock information PCR on that program is relative to the time information PTS in the decoder 17. 
included. 20 j Q ^ conve ntional digital broadcasting system 1 con- 
The receiving device 3, referring to the program infer- structed as described above, video and audio data for mul- 
mation PMT described above, extracts the TS packet in trp i e programs can be transmitted. However, as to each 
which the clock information PCR is included. In the program, only one video data can be transmitted. Even in 
example shown in FIG. 8, since the packet identification one program, if a plurality of video data associated with the 
information PID of the clock information is "XC", the TS 2 5 program can be transmitted, it is considered that the usability 
packet having the packet identification information PID G f lrie digital broadcasting system can be further improved. 
"XC" is extracted by the demultiplexer 13. The receiving For examp i e , in the case where the program A is a concert 
device 3 has a counter (STC) 15 which conducts the count- program of a prescribed musician, if the low angle video 
ing operation at 27 MHz clock CLK2 generated in the clock data obtained by photographing the concert from the lower 
generator 14, and sets the clock information PCR included 30 partj the m iddle angle video data obtained by photographing 
in the extracted TS packet to the counter (STC) 15 as an tne concert from the front, and the high angle video data 
initial value, and makes the counter (STC) 15 to count obtained by photographing the concert from the upper part 
operate based on the clock CLK2. Then, obtaining the can be transmitted as the video data of the program A, the 
difference between the clock information transmitted at a user can selecl desired video data among the plurality of 
fixed cycle and the count value of the counter (STC) 15, and 35 v i deo data of the program A to view the program that fits to 
by controlling the operation of the clock generator 14 by a nis tastCt it ^ considered that the usability can be 
phase locked loop (PLL) 16 based on the difference, the further improved. 

receiving device 3 forms a clock CLK2 of which phase and In order t0 transmit a plurality of video data for one 
frequency completely conform with that of the clock CLK1 prog ram, the conventional digital broadcasting system 1 
generated in the transmitting side. Thus, by transmitting the 40 tfeats the plurality of video data which are originally one 
clock information PCR from the transmission device 2 and pr0 gram, as separate programs, and multiplexes and trans- 
controlling the operation of the clock generator 14 based on mits them as the separate programs respectively. For 
the clock information PCR in the receiving device 3, the example, if the video data of middle angle program A is 
clock CLK2 having the same frequency and phase as the transm i tted as a program A, the video data of low angle 
transmitting side can be reproduced m the receivmg device 45 pro gram A is transmitted as a program B, and the video data 
3 and the operation of the counter (STC) 15 can be syn- of high angle progfam A is transmiUed as a program c> 
chromzed with the operation of the transmitting side. desired video dala can ^ elected according to the principle 

In the transmission device 2 of the transmitting side, in the 0 f program selection described above. However, if these 

case of forming the PES packet, the time information PTS were treated mer ely as the separate programs, when shifting 

to regulate the timing of reproduction and output of video or so the video data from middle angle to high angle video data, 

audio data contained in the PES packet is added. And the it ^ necessary to obtain the program information PMT 

decoding processing is conducted at the timing based on the whenever the video data are shifted because these are 

time information PTS, in order to synchronize video and different programs. Therefore, the reception processing 

audio without excess or lack of data. Accordingly, the becomes complicated, and there is a possibility that it cannot 

receiving device 3 takes out the time information PTS added 55 be shifted to another video data immediately. Moreover, in 

when PES packetizing, from the TS packet received, com- tne case wnere tne p i ura iit y of data are treated as separate 

pares the time information PTS to the value of its own programs, programs cannot be synchronized with each other, 

counter (STC) 15, and starts the decoding processing at the XhuSj there ^ a possibility that video data does not continue 

timing when these values coincide. wnen tDe ^0 data ^ shifted. 

For example, in the case of decoding video data of the 60 citmmarv hp thf iNVFNrrrnM 

program in the example shown in FIG. 8, firstly the demul- SUMMARY OF THE INVENTION 

tiplexer 13 takes out the TS packet having the packet In view of the foregoing, an object of this invention is to 

identification information PID "XV" from the transport provide a video transmission device and its method capable 

stream received, and restores PES packet from the TS packet of transmitting a plurality of video data associated with one 

and transmits this to the decoder 17, as shown in FIG. 11. 65 program without a complicated reception processing. 

Moreover, the demultiplexer 13 transmits the PES packet to The foregoing object and other objects of the invention 

the PTS receiving unit 18. The PTS receiving unit 18 have been achieved by the provision of a video transmission 
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device and its method. In the present invention, a plurality DETAILED DESCRIPTION OF THE 

of associated video data is encoded respectively and then, EMBODIMENT 

different packet identification information is added to each Preferred embodiments of this invention will be described 

coded video data to packetize the coded video data. And in ^ reference to the accompanying drawings: 

order to control the plurality of associated video data as one 5 , n nG 12 2 „ M shows , transmission devic6 of 

program, first program information to show the packet a broadc * ting system according to the present 

identification information added to coded video data invemion> which is to ^ ansmit a plurality of ^ciated ^ 

respecuvely, per video data is formed, and proper packet ^ ag one More fficall m 

identification information is added to the first program . VL9 K M . A vrt . * A , ° , . • i_ . . ~ 

n . • r the transmission device 20, low angle video data S^, 

mformation to packetize the first program information. 10 o5t ^ ned 5y photogra hing a concert by a preS cribed musi- 

Then by multiplexing packets concerning each video data ^ frQm lo ^ e / r r (hereinafter , referred t0 „ first 

and the first program mformation first transmission data or ^ da fa emered ^ o a fifst yideo encodm ^ ^ and 

one program is formed. Further, the first transmission data, ^ data $ b hotographing the 

the second transmission data formed of a different program, . r .U * ♦ /u ^- ^r-. ^ 't/„~A Z ^ *JZ„„a 

. , . - . . | • i concert from the front (hereinafter, referred to as second 

and second program information to snow the packet iden- 15 . , « . . . . A . ; , . , „ ~~ 

. / 6 . «,j • /j , . r 4 . video data) is entered into a second video encoding unit 22, 

tification information added to the first program mformation , . 7 , , c j u „ 

. , • j and high angle video data S™ obtained by photographing 

of each program to be transmitted per program are multi- ^ ^ w ereiMfte teferred (o ^ 

plexed in order to form and transmit third transmission data ^ ^ ^ fc ^ a v ^ ^ 6ncodin ^ 

through the predetermined circuit. ^ and ^ ^ ^ accompanying lhe ^ data Svi> Svjvf 

In this way, m the case of transmitting a plurality of 2° or Sw ^ emered mto an audio encoder 24. 

associated video data, since the first program information for In , he firs , videQ encoding wit 2 i, firstly the first video 

controlling the plurality of associated video date as one data s Ued fc en(ered mto a videf) encoder 21j(Sl ^ 

program is formed and transmitted, video data can be shifted video encoder 21A successively compress-encodes the first 

without receiving program information whenever shifting video data based on the MPEG2 video encoding system 

video data, like the conventional case in which associated 25 fa tbe International standards Regulation ITU-T 

video data are transmitted as separate programs. Thus, the Recommendalion H .262 ISO/1EC13818-2, and outputs a 

processing of the receiving side can be simplified. stream of me resultant <Joded video daU §21 , 0 , packetizer 

The nature, principle and utility of the invention will 21B. 

become apparent from the following detailed description i n mis connection, the video encoder 21A conducts the 

when read in conjunction with the accompanying drawings encoding processing with video data for several pictures 

in which like parts are designated by like reference numerals which are called GO p ^ a m ding unit in the case of 

or characters. encoding the first video data S^, similarly to the case shown 

BRIEF DESCRIPTION OF THE DRAWINGS in F U IGS - 3A and 3 f B ' and h * ^ing a header "j^ing of 

35 each parameter information at the time of encoding to the 

In the accompanying drawings: head of the coded video data, forms coded video data S21. 

FIG. 1 is a block diagram showing the construction of a The packetizer 21B, similarly to the case shown in FIGS, 

digital broadcasting system; 4A and 4B, connecting a fixed number of ES packets of the 

FIG. 2 is a block diagram showing the construction of a coded video data S21 encoded per GOP and adding a PES 

conventional transmission device; 40 header to its head, PES-packetizes the coded video data S21 

FIGS. 3A and 3B are schematic diagrams explaining the ™ d out P uts the PES-packetized coded video data S22 to a 

video data encoding- packetizer 21 C. In this case, the PES header includes 

™™ *L * ,J; * . A . i • * t. time information PTS for reproducing and outputting the 

FIGS. 4A and 4B are schematic diagrams explaining the ^ data a( ^ feceivi gide 

video data PES packetization; . , ncc . . - 

, . ' . ... 45 The TS packetizer 21C divides each PES packet of the 

FIGS. 5A and 5B are bnef linear diagrams explaining the coded yideo data s22 lg4 b tes similar tQ the e k 

TS packet formmg method; sh()wn - n nG 5) and forms a TS packets by adding a ^ 

FIGS. 6A and 6B are timing charts showing the timing of header of 4 bytes to each of their heads, and outputs a stream 

TS packet; 0 f me t§ packets S23 to a multiplexer 25 of the succeeding 

FIG. 7 is a table explaining program information PAT; 50 stage. Also in this case, the TS header is formed of syn- 

FIG. 8 is a table explaining program information PMT; chronization information and packet identification informa- 

FIG. 9 is a schematic diagram explaining an algorithm of tion PID. In this connection, this TS packetizer 21C, by 

program selection; adding the packet identification information PID whose 

FIG. 10 is a block diagram explaining the clock recovery value » " XVr lo the J$ P«= kel shows the TO 

method in the receiving device; 55 packet is associated with the first video data. 

11 • ui i j- i * * *u a a- . w Similarly, in the second and the third video encoding units 

FIG. 11 is a block diagram explaining the decodmg timing ^ t J" J \ , , , .. . , ^ 6 . ^ 

4i *u a • *u • • a ■ ~ 22 and 23, the second and the third video data and S™ 

control method in the receiving device; > , 11A , Y?k * c 

■ i_i i j- Z i_ 4 t . c are encoded per GOP by video encoder 22A and 23A to form 

FIG. 12 is a block diagram showing the construction of a coded video F data s2 / and s27j respectively. The coded 

transmission device of a digital broadcasting system accord- 6Q ^ ^ §24 and s2? ^ PES - P acketized by following TS 

mg to the present invention; packetizers 22C and 23B to form coded video data S25 and 

FIGS. 13A and 13B are schematic diagrams explaining s2g) reS p eclivelVi the coded video data S25 and S28 

GOP sequence of each video data; are TS-packetized by following TS packetizers 22C and 23C 

FIG. 14 is a schematic diagram showing program infor- respectively, and the resulting TS packet streams S26 and 

mation PMT to be generated in the transmission device; and 65 S29 are sent out to the multiplexer 25 of the succeeding 

FIG. 15 is a block diagram showing the construction of a stage. The TS packetizers 22 C and 23C, by adding the 

receiving device. packet identification information PID having values "XV2" 
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and "XV3" to the TS packets formed respectively, shows 
that the TS packets are in associated with the second and the 
third video data S w and S w respectively. 

On the other hand, in the audio encoding unit 24, the 
audio data S A supplied is firstly entered into an audio 5 
encoder 24A. The audio encoder 24A encodes the input 
audio data S A based on the audio encoding system described 
in the International Standards Regulation ITU-T Recom- 
mendation H.262 ISO/IEC13818-3, and outputs the result- 
ant coded audio data S30 to a packetizer 24B. Also, the 10 
audio encoder 24A encodes the audio data S A per fixed 
number of samples that is called an audio frame, similarly to 
the case of video data shown in FIGS. 3A and 3B, and by 
adding a header consisting of each parameter information at 
the time of encoding to its head, forms ES-packetized coded 15 
audio data S30. 

The packetizer 24B, by connecting a fixed number of ES 
packets of the coded audio data S30 encoded per audio 
frame and adding a PES header to the head of its packet, 
PES-packetizes the coded audio data S30 and sends out the 20 
PES-packetized coded audio data S31 to a TS packetizer 
24C 

The TS packetizer 24C divides each PES packet of the 
coded audio data S31 per 184 bytes as in the case shown in 25 
FIGS. 5A and 5B, and by adding a TS header of 4 bytes to 
each of their heads, forms a TS packet, and transmits a 
stream of the TS packets S32 to the multiplexer 25 of the 
succeeding stage. Also in this case, the TS header consists of 
synchronization information and packet identification infor- 3Q 
mation PID. In this connection, the TS packetizer 24C, by 
adding the packet identification information PID having the 
value "XA" to the TS packet formed, shows that the TS 
packet is associated with the audio data S A . 

An encoding control unit 26 provided in the transmission 35 
device 20 controls the overall operations of the video 
encoders 21A to 23A and packetizers 21B to 23B of the 
video encoding unit 21 to 23 and the audio encoder 24A and 
the packetizer 24B of the audio encoder 24. More precisely, 
the encoding control unit 26 can make these circuit start all 40 
at once by transmitting a control signal S33 to the video 
encoders 21 A to 23A, the audio encoder 24A and the 
packetizers 21B to 24B. 

Furthermore, the encoding control unit 26 sets the video 
encoders 21A to 23A so that their parameters at the time of 45 
encoding become the same, using the control signal S33. 
More specifically, as shown in FIGS. 13 A to 13C, the 
encoding control unit 26 sets the video encoders 21A to 23A 
so that the cycle M of P-picture and cycle N of I -picture at 
the time of encoding become the same, and simultaneously, 50 
so that the phases of GOPs to video data S^, S w , and S VH 
synchronize with each other. Moreover, the encoding control 
unit 26 sets the GOP to be used in each of the video encoders 
21 A to 23A so that the reference relationship of pictures is 
completed within each GOP, that is, becomes a closed GOP. 55 
To be more precise, the closed GOP, as shown in FIGS. 13A 
to 13C, in the case of encoding B-picture placed at the head 
of each GOP (B-picture having the picture numbers 4 and 5 
in this example), the encoding by the forward direction 
prediction that makes P-picture existing in the preceding 60 
other GOP as the reference picture is not conducted, but the 
encoding by the backward direction prediction that makes 
I-picture existing in the same GOP as the reference picture 
is conducted. Accordingly, by conducting the encoding 
processing in the closed GOP, if the video data S^ is shifted 65 
to the video data S^ at the lime point tl in the receiving 
device side, for example, since the reference picture exists in 
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the same GOP, the failure in reproduced picture can be 
prevented in advance. 

Furthermore, the encoding control unit 26, by transmit- 
ting the control signal S33 to the packetizers 2 IB to 13B, 
controls the PES packetizing processing so that the head of 
data part of the PES packet becomes GOP (i.e., the GOP 
does not cross over different PES packet), and 
simultaneously, the encoding control unit 26 assigns the time 
information PTS to the packetizers 2 IB to 23B referring to 
the count value of the counter (STQ 11 so that the time 
information PTS to be added at the time when PES pack- 
etizing become equal among the first to third video data S^, 
S VM and S w . 

In this connection, since the audio has the encoder dif- 
ferent from that of the video, the encoding control unit 26 
originally controls the audio encoder 24A and the packetizer 
24B. 

The counter (STC) 11 conducts the counting operation at 
27 MHz clock CLK1, that is the reference clock for the 
encoding processing. The count value of this counter (STC) 
11 is transmitted to a TS packetizer 27 at a fixed cycle as 
clock information PCR. The TS packetizer 27 successively 
TS-packetizes the clock information PCR transmitted at the 
prescribed cycle, and transmits a sequence of the TS packets 

534 to the multiplexer 25. In this case, the TS packetizer 27, 
adding the packet identification information PID having the 
value "XC" to the TS packet formed, shows that the TS 
packet is associated with the clock information PCR. 

A PMT forming unit 28A is program information forming 
means for forming program information PMT of lower 
hierarchy arranged per program, in which the program 
information PMT for one program consisting of the first or 
the third video data S^r, Sy^ and S w and the audio data S A 
and the clock information PCR is formed. The program 
information PMT, as shown in FIG. 14, assuming that a 
program number of the program is "X", the packet identi- 
fication information PID of TS packet in which data com- 
prising the program is stored, per each data. More 
specifically, the packet identification information PID of the 
TS packet in which the low angle first video data 
stored is "XVI", and the packet identification information 
PID of the TS packet in which the middle angle second 
video data S w is stored is "XV2". Further, the packet 
identification information of the TS packet in which the high 
angle third video data S w is stored is "XV3", the packet 
identification information of the TS packet in which the 
audio data S A is stored is "XA", and the packet identification 
information PID of the TS packet in which the clock 
information PCR is stored is "XC". 

The PMT forming unit 28 A, after forming the program 
information PMT having such data, packetizes the informa- 
tion and sends out a sequence of the resultant TS packets 

535 related to the program information PMT to the multi- 
plexer 25. The PMT forming unit 28 A adds the packet 
identification information PID having the value "XX" to the 
TS packet formed, in order to show that the TS packet is 
related to the program information PMT. 

The multiplexer 25 is multiplexing means, in which by 
sequentially selecting and outputting TS packet streams S23, 
S26, S29 S32, S34 and S35 to be supplied from circuit 
blocks respectively in a certain order, one multiplexed 
transport stream S3 6 for one program is formed and trans- 
mitted. Note that, buffer circuits not shown are provided in 
inputs of the multiplexer 25 respectively. The buffer circuits 
temporarily store the TS packet sequences S23, S26, S29, 
532, S34, and S35 respectively, in order to selectively output 
them in a prescribed order. 
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In the second multiple xer 29, transport streams S37 to S39 S53 of the program information RAT is sent to the PAT 

concerning the other programs formed with the same storage unit 35. The PAT storage unit 35 analyzes the TS 

method as the transport stream S36 are entered. Moreover, packet S53 and stores program information included in it in 

a TS packet sequence S40 concerning the program infor- the internal memory area. 

mation PAT formed at the PAT forming unit 28B is also 5 Under such conditions, at the time of requesting a desired 

entered into the second multiplexer 29. program from the user with the operation unit 36, the control 

In this case, the PAT forming unit 28B is the program unit 34 firstly refers to the program information PAT stored 

information forming means for forming the program infor- in the PAT storage unit 35 based on the request. Then, the 

mation PAT of the higher hierarchy showing the TS packet control unit 34 obtains the packet identification information 

in which program information PMT formed per program are 1° PID of the TS packet storing the program information PMT 

stored, in which program information PAT on all programs of the requested program, and making the demultiplexer 33 

to be multiplexed by the second multiplexer 29 are formed. extract the TS packet based on that packet identification 

The program information PAT are arranged according to information PID, in order to extract the TS packet S54 

program numbers similarly to the example shown in FIG. 7; including program information PMT of the requested pro- 

the packet identification information PID of the TS packet 15 gram. The TS packet S54 of the program information PMT 

storing program information PMT on the program number is sent to the PMT storage unit 37. 

"0" is "N", the packet identification information PID of the ^ PMX slora ge unit 37 analyzes the TS packet S54 and 

TS packet storing the program information PMT concerning slores the p rogram information PMT included in the TS 

the program number "1" is "A", and the packet identification packet in the mte mal memory area. The control unit 34 

information PID of the TS packet storing the program 20 obtains the packet identification information PID storing 

information PMT concerning the program number "X" is data forming the requested at program referring to the 

"XX". program information PMT stored in this PMT storage unit 

The PAT forming unit 28B, after forming the program 37. In this case, a plurality of video data exist per one 

information PAT having such data, TS-packetizes this and program and the packet identification information PID 

transmits the resultant TS packet sequence S40 concerning 25 related to the video data S VM is obtained from the PMT 

the program information PAT to the second multiplexer 29. storage unit 37 as the initial value. 

The PAT forming unit 28B adds the packet identification unit 34 makes tne demultiplexer 33 extract 

information PID having the value "0" to the TS packet tne TS pac k et b asec j 0 n the packet identification information 

formed, in order to show that the TS packet is related to the PID obtained, in order to extract TS packets S55 to S57 

program information PAT. 30 storing video data S w , audio data S A and clock information 

The second multiplexer 29, by successively selecting and PCR of the requested program respectively. Out of the TS 

transmitting the transport stream S36 on the program num- packets extracted, the TS packet S55 related to the video 

ber "X" and the transport streams S37 to S39 concerning the data S VM is transmitted to a video decoding unit 38 and the 

other programs and the TS packet sequence S40 related to ^ TS packet S56 related to the audio data S A is sent out to an 

the program information PAT in a prescribed order, forms a audio decoding unit 39, and the TS packet S57 related to the 

transport stream S41 obtained in by multiplexing a plurality clock information PCR is transmitted to a counter (STC) 15. 

of programs into one stream, and outputs the stream S41 to counter (STC) 15 takes out clock information PCR 

the transmission unit 9 of the succeeding stage. included in the TS packet S57 and sets this as the initial 

The transmission unit 9, after forming a transmission 4Q value of the counter and starts the counting operation based 

signal by applying the modulation processing for transmit- on 27 MHz clock CLK2. The phase-locked loop (PLL) 16 

ting to the transport stream S41, forms a transmission signal obtains the difference between the clock information PCR 

S42 having a prescribed band by applying a frequency included in the TS packet S57 to be transmitted at the fixed 

conversion processing to the transmission signal, and trans- intervals and the count value of the counter (STC) 15, and 

mits the signal S42 toward the satellite (not shown) via an 45 by controlling the clock generator 14 based on the 

antenna 10. difference, generates a clock CLK2 synchronized with the 

Then, the receiving device of the digital broadcasting clock CLK1 of the transmission device 20, in order to 

system for receiving the transmission signal S42 will be synchronize the counting operation of the counter (STC) 15 

described referring to FIG. 15. The transmission signal S42 with the counter (STC) 11 of the transmission device 20. 

relayed via the satellite is received by an antenna 31 of the 50 Thus, the count value of the counter (STC) 15 synchronized 

receiving device 30 shown in FIG. 15, and the received with the transmitting side is sent out to a decoding control 

signal S50 is supplied into the receiving unit 32. The unit 40 for controlling the reproduction output timing, 

receiving unit 32, by applying the frequency conversion The video decoding unit 38 firstly enters a TS packet S55 

processing and the demodulation processing to the received on the video to be supplied, into an internal ES streamizer 

signal S50, restores a transport stream S51 to be transmitted 55 38A. The ES streamizer 38 A restores the PES data by 

from the transmission device 20 and sends this out to a connecting the video data to be supplied after being 

demultiplexer 33 and a control unit 34. TS-packetized, and further converts the PES data to an 

The control unit 34 identifies the packet identification elementary stream S58 per coding unit and outputs the 

information PID of each of the TS packets included in the elementary stream S58 related to the video to the video 

transport stream S51, and outputs a control signal S52 based 60 decoder 38B. Moreover, the ES streamizer 38 A takes out the 

on the packet identification information PID to control the time information PTS concerning the video included in the 

selecting operation of the demultiplexer 33. Thereby, the PES data when converting the TS packet S55 into the PES 

desired TS packet is extracted from the transport stream S51. data, and outputs this to the decoding control unit 40 as a 

More specifically, the control unit 34 makes the demul- time signal S59. 

tiplexer 33 extract the TS packet having the packet identi- 65 The video decoder 38B starts the decoding processing for 

fication information PID value "0" and take out the TS the elementary stream S58 based on the timing information 

packet S53 of the program information PAT The TS packet S60 supplied from the decoding control unit 40, and sends 
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out the resultant restored video data S61 to the television 
receiver (not shown) of the succeeding stage. 

Similarly, at first, the audio decoding unit 39 supplies the 
TS packet S56 concerning audio supplied, to the internal ES 
streamizer 39A. The ES streamizer 39 A restores the PES 
data by connecting audio data to be transmitted after being 
TS-packetized, further converts the PES data into an 
elementary stream S62 per the coding unit, and outputs the 
elementary stream S62 concerning the audio to the audio 
decoder 39B. Moreover, the ES streamizer 39A takes out the 
time information PTS concerning audio included in the PES 
data at the time of converting the TS packet S5<Linto the PES 
data, and outputs this to the decoding control unit 40 as a 
time signal S63. 

The audio decoder 39B starts the decoding processing for 
the elementary stream S62 based on the timing information 
S66 supplied from the decoding control unit 40, and outputs 
the resultant restored audio data S64 to the television 
receiver of the succeeding stage. 

The decoding control unit 40 judges whether timing of the 
time information PTS shown by time signals S59 and S63 
have come or not, based on the time signals S59 and S63 
concerning video and audio and the count value of the 
counter (STC) 15. When it judges that timing has come, the 
timing information S60 is transmitted to control the decod- 
ing start of the video decoder 38B and the audio decoder 
39B. 

Accordingly, the receiving device 30 obtains the program 
information PMT of the program requested by the user, 
referring to the program information PAT and then, referring 
to the program information PMT, takes out the TS packet of 
data forming the requested program and restores video and 
audio data S61 and S64 of the program. 

By the way, a plurality of associated video data S^, S VAf 
and S VH are transmitted from the transmission device 20 for 
one program. In the aforementioned explanation, the video 
data is selected as the initial value, however, in the case 
where the user requests to switch to the other video data S^ 
or S w , these video data can be shifted easily. 

For example, in the case where a request to switch to the 
video data S w is entered with the operation unit 36, the 
control unit 34 obtains the packet identification PID of the 
TS packet storing the video data S w , referring to the 
program information PMT stored in the PMT storage unit 
37, based on the request. The control unit 34 makes the 
demultiplexer 33 extract the TS packet based on the packet 
identification information PID to take out the TS packet of 
the video data S w . The TS packet S54 extracted is sent to 
the video decoding unit 38 to be decoded, and is transmitted 
as video data S61. Thus, it can easily switch to the video data 
Syn requested. 

In this connection, in the case of switching the TS packet 
to be extracted at the demultiplexer 33, the control unit 34 
switches it at the timing of the following PES data. In this 
case, as well as matching the head of PES data of each video 
data S^, S VM and S VH to the head of GOP, the phases of 
GOPs of video data S^, S m and S VH Synchronize with 
each other. Furthermore, since individual GOP is set to the 
closed GOP, data failure never occurs in the reproduced 
picture even though it is switched to another video data. 
Moreover, since time information PTS and clock informa- 
tion PCR are available in he same program, even when it is 
shifted to another video data, the phase of the reproduced 
picture can be prevented from shifting and the continuity of 
the reproduced picture can be maintained. 

According to the foregoing construction, the transmission 
device 20 of the digital broadcasting system TS-packetizes 
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and transmits a plurality of associated video data S^, S^ 
and S v// . In this case, the transmission device 20, adding 
different identification information PID to the TS packet of 
each of the video data S^, S^ and S w transmits them. 
5 Moreover, the transmission device 20 forms the program 
information PMT to control the associated video data S^, 
S VM and Syjf as one program, and also TS-packetizes and 
transmits the program information PMT. Moreover, in the 
transmission device 20, the program information PAT of the 
higher rank to show the TS packet storing the program 
information PMT of the program is also TS-packetized and 
transmitted. 

Accordingly, in the transmission device 20, 
TS-packetized video data and program information PMT 
and PAT of each program are finally multiplexed to one 

15 transport stream S41 and transmitted to the receiving device 
30 through the satellite wireless circuit. In the receiving 
device 30, firstly, the program information PAT of the higher 
rank is received, and referring to the program information 
PAT, the TS packet storing the program information PMT of 

20 the program requested by the user is received to obtain the 
program information PMT of the program requested by the 
user. And the receiving device 30, referring to the program 
information PMT, receives the TS packet of the middle angle 
second video data S VM that is set as the initial value, and 

25 conducting the decoding processing, outputs the second 
video data S VM . 

Under such conditions, if a command to switch to another 
video data S^ or S w is entered by the user, the receiving 
device 30 receives the TS packet of the requested video data 

30 S^ or Syff referring to the program information PMT stored 
in the PMT storage unit 37 and decoding this, in order to 
output the requested video data S^ or S^. 

In the case of this digital broadcasting system, a plurality 
of associated video data Sy^, S VAf and S VH are 

35 TS-packetized and transmitted respectively and 
simultaneously, the program information PMT to control the 
plurality of associated video data S^, S^ and S w as one 
program is TS-packetized and transmitted, so that the receiv- 
ing side can easily switch the video data S^, S w and S w 

40 referring to the program information PMT, once the receiv- 
ing side receives the program information PMT. In the case 
where associated video data S^, S w and S are trans- 
mitted as separate programs as before, it is necessary to get 
the program information PMT whenever switching video 

45 data because the program information PMT is formed per 
program. Accordingly, there was that the processing of the 
receiving side became complicated. However, if the pro- 
gram information PMT to control the associated video data 
Syz, S w and S w as one program is transmitted, all packet 

50 identification information PID of the associated video data 
$vl> anc * S v// can be grasped and thus, such problem can 
be solved. 

Furthermore, in the case of encoding the plurality of 
associated video data S^, S^ and S w , the encoding 

55 processing to synchronize the coding unit at the time of 
encoding between these video data S^, S VM and S VH within 
the coding unit is conducted, so that the failure of the 
reproduced picture when switching the video data S^, S w 
and Syfj can be prevented. 

60 Furthermore, in the case of encoding the associated video 
data S^, S VM and S w , the timing information PTS to 
regulate the timing at the time of reproduction-output are to 
become equal among the video data S^, S VM and S w , the 
phase of the reproduced picture can be prevented from 

65 shifting when switching the video data S^, S w and S^ 
and the continuity of the reproduced picture can be main- 
tained. 
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According to the foregoing construction, since a plurality 
of associated video data S v/ _, S VM and S Vff are 
TS-packetized and transmitted, and simultaneously, the pro- 
gram information PMT to control the plurality of associated 
video data S VL , S VM and S VH as one program is 5 
TS-packetized and transmitted. Thereby, a plurality of video 
data S^, S VM and S w associated with one program can be 
easily transmitted without conducting a complicated pro- 
cessing. 

Note that, in the aforementioned embodiment, a plurality 1Q 
of associated video data S^, S VM and S VH is transmitted as 
one program. However, the present invention is not only 
limited thereto and a plurality of associated audio data can 
be transmitted as one program. In the case of transmitting 
the plurality of associated audio data as one program, 
similarly to the case of the video data, a plurality of 
associated audio data is encoded and TS-packetized 
respectively, TS packets of the plurality of associated audio 
data are multiplexed and transmitted, and program informa- 
tion PMT to control the associated audio data as one 
program is TS-packetized and transmitted. Thereby, the 20 
same effects as those of the video data can be obtained. 

Moreover, in the aforementioned embodiment, an account 
method is not described. However, video data S^, S w and 
S VH are scrambled using a secret key respectively, and the ^ 
receiving side removes the scramble using the secret key 
corresponding to the received video data, so that fee can be 
charged per each secret key used. With this arrangement, the 
fee can be charged per video data even though video data are 
the same one program. Moreover, fee can be charged per 3Q 
program by applying the scramble to video data S^, S VM 
and S v/f using the same secret key. 

Furthermore, in the embodiment described above, the 
video data S w out of associated video data S^, S VM and 
Syjj is reproduced and outputted as the initial value. 35 
However, the present invention is not only limited thereto 
and another video data S^ or S VH can be transmitted as the 
initial value. 

Furthermore, in the embodiment described above, video 
data Svl, S VM and S w each having the different photo- 40 
graphic direction are transmitted as the associated video 
data. However, the present invention is not only limited 
thereto and main video data, and sub video data to provide 
supplemental remarks to the main video data, such as by the 
sign language can be transmitted. Thereby, the same effects 45 
as those of the above can be obtained. 

Moreover, in the embodiment described above, the trans- 
port stream S41 formed finally is transmitted through the 
satellite wireless circuit. However, the present invention is 
not only limited thereto this and a transport stream can be 50 
transmitted through a terrestrial wireless circuit and a cable 
circuit. In short, the present invention can be widely applied 
as long as the transmission device transmits a finally formed 
transport stream through a prescribed transmission path. 

According to the present invention as described above, in 55 
the case of transmitting a plurality of associated video data, 
the first program information to control the plurality of 
associated video data as one program is formed and 
transmitted, so that a plurality of video data associated with 
one program can be transmitted without a complicated 60 
processing at the time of reception. 

While there has been described in connection with the 
preferred embodiments of the invention, it will be obvious 
to those skilled in the art that various changes and modifi- 
cations may be aimed, therefore, to cover in the appended 65 
claims all such changes and modifications as fall within the 
true spirit and scope of the invention. 



What is claimed is: 

1. A video transmission device comprising: 

a plurality of video encoding means for encoding a 
plurality of associated video data and for adding dif- 
ferent packet identification information to the coded 
video data respectively to packetize and output them; 

program information forming means, in order to control 
said plurality of associated video data as one program, 
for forming first program information to show said 
packet identification information, which are added to 
said coded video data respectively, per video data and 
for adding proper packet identification information to 
the first program information to packetize and output it; 

first multiplexing means for multiplexing packets con- 
cerning each of video data to be transmitted from said 
plurality of video encoding means onto packets of said 
first program information to be transmitted from said 
program information forming means, and for output- 
ting first transmission data for one program; 

second multiplexing means for multiplexing said first 
transmission data to be transmitted from said first 
multiplexing means and second transmission data 
formed of another program onto second program infor- 
mation showing said packet identification information, 
which is added to said first program information of 
each program to be transmitted, per program, and for 
outputting third transmission data; and 

transmission means for transmitting said third transmis- 
sion data to be transmitted from said second multiplex- 
ing means through a prescribed circuit. 

2. The video transmission device according to claim 1, 
further comprising 

a plurality of audio encoding means for encoding a 
plurality of associated audio data and for adding dif- 
ferent packet identification to the coded audio data 
respectively to packetize and output them; wherein 
said program information forming means, as well as 
showing said packet identification information, 
which are added to said coded video data 
respectively, per video data, forms said first program 
information showing said packet identification 
information, which are added to said coded audio 
data respectively, per audio data. 

3. The video transmission device according to claim 1, 
wherein 

said plurality of video encoding means, as well as encod- 
ing video data so that coding units synchronize between 
the video data, applies such an encoding as finishes 
within the coding unit, to said video data. 

4. The video transmission device according to claim 1, 
wherein 

said plurality of video encoding means, as well as adding 
time information to regulate a processing timing at a 
decoding side, to each of said coded video data, makes 
the time information to be equal among the video data. 

5. A video transmission method, comprising the steps of: 
as well as encoding a plurality of associated video data, 

adding different packet identification information to the 
coded video data respectively to packetize them; 
in order to control said plurality of associated video data 
as one program, forming first program information to 
show said packet identification information, which are 
added to said coded video data respectively, per video 
data and adding proper packet identification informa- 
tion to the first program information to packetize it; 
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forming first transmission data for one program by mul- 
tiplexing packets concerning said video data onto pack- 
ets for said first program information; and 

forming third transmission data by multiplexing said first 
transmission data, second transmission data formed of 5 
another program onto second program information 
showing said packet identification information, which 
is added to said first program information of each 
program to be transmitted, per program, and transmit- 
ting the third transmission data through a prescribed 10 
circuit. 

6. The video transmission method according to claim 5, 
wherein: 

a plurality of associated audio data is encoded, and 
different packet identification information are added to 35 
coded audio data respectively to packetize the coded 
audio data; and 

said packet identification information, which are added to 
said coded video data respectively, are showed per 
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video data, and said first program information showing 
said packet identification information, which are added 
to said coded audio data respectively, per audio data is 
formed. 

7. The video transmission method according to claim 5, 
wherein 

encoding is applied to video data so that coding units 
synchronize between the video data, and such an 
encoding as finishes within the coding unit is applied to 
said video data to respectively. 

8. The video transmission method according to claim 5, 
wherein 

in the case of encoding said video data, time information 
to regulate a processing timing of a decoding side is 
added to each of said coded video data, and the time 
information is to be equal among the video data. 
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